Networks such as business networks may include a home or headquarters node, and field office or branch office nodes. Such nodes may communicate via a wide area network (WAN) such as the Internet, for example. Large amounts of computing resources and transmission resources may be expended by redundant data transmission, for example, between a branch office and a headquarters office. WAN optimization products may involve using a physical branch office appliance, for example, to determine whether requested data items have already been requested and received at the branch office (and thus may be retrieved from a cache memory located at the branch office instead), in efforts to eliminate or minimize redundancy of WAN transmissions. Such a physical branch office appliance may involve associated costs of deployment and maintenance of the appliance. Further, boundaries of branch offices are increasingly becoming less clearly defined, as many businesses are moving toward shared office space and telecommuting environments.
Additionally, some techniques for WAN redundancy elimination are experiencing scaling problems in datacenters or headends, for example, in the home or headquarters offices, where a number of branch office connections may be aggregated. Redundancy elimination may be performed more efficiently in environments supporting more storage for caching redundant chunks of data on both sides of the WAN (i.e., at the headquarters office and at the branch office). For example, if an enterprise includes 1,000 branch offices, and each branch office includes one gigabyte (GB) of cache memory available for storage, then the headquarters office may need 1,000*1 GB=1 TeraByte of cache memory to be able to process the information associated with all the branch offices. Disk based cache memory may be implemented to achieve the needed size of memory; however, disk based cache may introduce various throughput limitations.